Generally, picture encoding standards such as MPEG of the International Standardization Organization (ISO) and H.263 recommended by the International Telecommunication Union (ITU) adopt block-based motion estimation and discrete cosine transform (DCT) blocks. When an image is highly compressed, the block-based coding may cause the well-known blocking effect. A typical blocking effect is grid noise in a homogeneous area in which adjacent pixels have relatively similar pixel values. Another blocking effect is staircase noise which has a staircase shape and is generated along the edges of the image. Also, ringing noise is a typical Gibb's phenomenon occurring by truncation when the coefficients of the DCT are quantized so as to highly compress the image.
Grid noise shows traces of the block-based process at the edges between blocks when the compressed data is displayed on a screen after being restored. Thus, one can identify the edges between blocks. Also, staircase noise has a staircase shape at the edges of the image, so that one can notice a bumpy edge on the image. Also, one can notice overlapping of images with a predetermined interval due to the ringing noise.
In order to reduce the blocking effect and the ringing noise occurring when block-based coding is performed, several methods have been suggested. According to H.261 encoding, a simple 3×3 low-pass filter (LPF) is used as a loop filter to reduce the blocking effect [“Video Codec for Audiovisual Services at P×62 kbit/s”, CCITT Recommendation H.261, Dec. 14, 1990]. Also, a simple edge loop filter has been suggested so as to reduce the blocking effect and mosquito noise [G.Bjontegaard, “A Simple Edge Loop Filter to Reduce Blocking and Mosquito Noise”, ISO/IEC JTCI/Sc29/WG11 MPEG96/0617, January, 1996, and “A Simple Edge Loop Filter to Reduce Blocking and Mosquito Noise”, ITU SCIS LBC Expert Group ITU-LBC-96-032, January, 1996]. The edge loop filter makes linear values of two pixels adjacent to the block boundary and replaces the two pixel values by the linearized values. Such edge loop filter can reduce the blocking effect but not the ringing noise. In order to reduce the ringing noise, a non-linear filter using a binary index has been suggested [Y. Itoh, “Detail Preserving Nonlinear Filter using Binary Index,” ISO/IEC JTC1/SC29/WG11 MPEG95/0357, November, 1995]. However, the non-linear filter cannot reduce the blocking effect.